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Abstract 

This paper studies the current evaluation system for the online training program in meteorology and hydrology. CIPP 
model that includes context evaluation, input evaluation, process evaluation and product evaluation differs from 
Kirkpatrick model including reactions evaluation, learning evaluation, transfer evaluation and results evaluation in that 
the subject of evaluation is different. We take the advantages of the CIPP model and Kirkpatrick model in constructing 
an evaluation system for online training programs in meteorology and hydrology held by the WMO Regional Training 
Centre Nanjing, China so as to improve the effectiveness of the training programs and meet the demand of the national 
meteorological and hydrological services. 
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1. The Current Status of the Evaluation of Online Training Programs 

Internet plays a more and more important role on the platform of training programs, because of its low cost and 
trans-regional advantages. During the period from 2000 to 2008, more than half of the world top 500 companies have 
adopted online training programs. For example, British Telecom has its 90 percent online training programs; IBM has 
established a college to provide its employees throughout the world with online training programs. The research on 
evaluation system for online training programs is more systematic in developed countries than in developing countries. 
On the aspect of evaluation tools, it can generally be divided into two main categories: the first one is online data 
collection tools; and the other is to record, analyze the online-time, frequency of login, pages that browsed by the 
employees who participate in these training courses. On the aspect of the construction of platform, there are already 
some well-developed web education platforms, taking the “Benchmarking Service” for instance, which is a professional 
website evaluating the enterprises’ training programs. On the aspect of evaluation for online training programs in China, 
it focuses on getting a certain kind of professional certificate after attending training course and passing the exam. On 
the aspect of constructing platform, some platforms have added evaluating function, but so far, the amount of the 
platforms which can sustain the systematic evaluation is still few. Most of the online training platforms mainly center on 
the level of online testing, following the process of the training course to analyze the changes in reaction and 
performance of trainees. 

There are already experts investigating how to evaluate the result of training, and they came out with a lot of evaluation 
models. Among these models, there are some models which center on the result of training, such as “Kirkpatrick 
Evaluation Model”, focusing on evaluating trainees’ changes in reaction both in the training process and after training. 
There are other models centering on training itself, such as “CIPP Model”. The paper aims at building the evaluation 
system for the online training programs. Only a systematic and integrated training schedule and procedure can 
guarantee the quality of training, so an efficient evaluation system should include a series of evaluation activities 
checking the training target, training itself and the result of training. Therefore, the paper takes the advantages of the 
“CIPP Model” and “Kirkpatrick Model” to construct a scientific and more effective evaluation system. 

“CIPP Model” is an evaluation model consisting of context evaluation, input evaluation, process evaluation and product 
evaluation. Context evaluation means defining environment and requirement, and carrying out demand analysis, which 
can help set up the training objective. Input evaluation is evaluation of the information or data which can be utilized to 
achieve the aim of training by the most effective approach. “Kirkpatrick Model” includes reactions evaluation, learning 
evaluation, transfer evaluation, and results evaluation. 
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2. Constructing the Evaluation Index System for Online Training Programs 

Employees of any profession need continuous study in work, so good training, such as online training program, can 
improve their comprehension towards new knowledge and technology, and eventually bring more rewards to the 
enterprise that they work for. The cost of online training is low, and the training program can reach more employees 
who want to share training resources. Therefore, to construct the evaluation system for the online training programs 
helps a lot to improve the quality of training. 

Comparing with traditional face to face study, the characteristics of online training is the division of time and space. 
Online training is implemented via reliable and safely network, trainees’ study is mainly autonomic learning which 
means that they carry out various kinds of learning activities on the platform supplied by web-based instruction 
independently. To ensure the process of study goes with a swing and finally the trainees achieve their goals. Meanwhile, 
we have to evaluate the level of guarantee of goal setting, supportive environment, resource development and utilization, 
the quality of teachers, the content and process of training, training environment and result, training objects and other 
items. 

The evaluation of goal setting, supportive environment, resource development and utilization, the quality of teachers, 
the content and process of training, training environment and result, training objects belongs to training itself, so we 
adopt “CIPP Evaluation Model”; meanwhile, we adopt “Kirkpatrick Evaluation Model”, which centers on the result of 
training, to evaluate training effectiveness. 

2.1 Constructing the Evaluation Index System Winch Centers on Training Subject 

The preferential item shall be the evaluation of context, which can be recognized as demand analysis. Numerous 
training programs put their emphasis on meeting the training demand during the whole course. They made their 
decision of carrying out an online training program relying on analysis of policy basis, implementation need, technical 
conditions, organization conditions, skills of trainees; and their training conspectus on the basis of practical demand. To 
some extent, the interaction during training and the satisfaction of trainees may reflect the quality of training. 

Following item is the evaluation of investment. Investment is a series of effort that training organization devotes in 
advance to achieve the target to make sure trainees get trained, such as supplying supportive environment, designing 
and constructing training resource, organizing teachers, setting out training curriculum, creating online training 
environment and so on. 

2.1.1 The Evaluation of Supportive Environment 

Supportive environment consists of layout plan, management system, financial support, technical support and facilities 
support. By the evaluation of background we can draw up a plan, and revise the plan according to reflection which 
comes out of the evaluation during training; the evaluation of management system is an indispensable link of the whole 
evaluation procedure, which is comprised of teaching management, assessment management, teacher management, 
resource construction management and so on. The evaluation of management system can undergo in the form of 
inspecting the conclusion documents or reports of the implementation conditions of every system, the evaluation of 
management system can promote the efficiency and stability of training in network environment. Online training is 
costly in early investment, resource construction and maintenance, so we have to make cost-benefit analysis, where the 
evaluation of financial support is essential. Expensive input may guarantee the quality of hardware and software of 
online training, but we shall pay enough attention to the balance of technical device input, make best use of resources 
and device. Technical support can be recognized as multi-media technology such as automatic Q & A system, auto 
homework system, and online study field, and intelligent platforms. Facilities support is mainly about constructing and 
optimizing hardware, making use of virtual laboratory, and constructing environment for practice. 

2.1.2 The Evaluation of Designing and Constructing Training Resources 

Evaluation of curriculum development is the core guarantee of the quality of the evaluation system. The quality of 
courses concerns a lot the success of the holistic training. The evaluation of curriculum development commences from 
design of curriculum, content of courses, direction, and demonstration of information. The design of curriculum shall 
adjust to the characteristics of self-study. As for self-study, study in coordination, practical activities, assessment and 
examination of different teaching activities, we should take full consideration, design the curriculum rationally, and 
widely popularize excellent courses. The evaluation of curriculum design consists of the evaluation of selection, 
adaption, or development of teaching material, academic environment design, instruction of trainees’ learning, content, 
homework arrangement and some other aspects. 

2.1.3 The Evaluation of Organizing and Arrangement of Instructors 

Instructors for online training should reach a certain level on professional knowledge, teaching experience and 
pedagogical psychology, they also have strong responsibility and a teaching organizing capacity, to give trainees advice 
in study, conduct study content, promote and maintain communication among trainees, so as to cultivate trainees’ 
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independence, enlighten trainees’ imagination, and problem-solving capability. The evaluation of instructors’ teaching 
quality could adopt different approaches, such as public hearing, evaluating lesson plan, teaching content, teaching style, 
teachers’ performance and participants’ attitude and so on. 

2.1.4 The Evaluation of Training Courses 

All kinds of courses are the vehicle of online training which consists of courseware, teaching activities and learning 
resources. The evaluation of training courses not only means to evaluate the results of implementation of courses, but 
also shall involve schedule, content, process of implementation and management of courses. 

2.1.5 The Evaluation of Training Network Environment 

The items that need evaluation include the interface’s individuation flexibility and some other items, like whether the 
administrator is aware of that a modification to navigation or reposting the attribute of a certain functional link may 
change navigational charts or backgrounds, as well as stability, cultural atmosphere, system operability, real-time, 
flexibility, expandability and efficiency. 

2.2 Constructing the Evaluation Index System Winch Centers on Trainees 

The evaluation of reactions usually carries out in a modest way to investigate trainees’ satisfaction after training. 
Furthermore, reaction evaluation may commence from trainees’ participation to evaluate trainees’ reaction, which 
means that if trainees enjoy their study with high enthusiasm, the training turns out to be an excellent one high quality 
and has its attractive merits. Participation may be inspected from trainees’ clicking rate, post and reply in forum, 
condition of homework submission, resource sharing and comment on each other’s work. 

Insert Table 1 Here 

The evaluation of learning is to examine and assess trainees’ learning effect, mastery of knowledge, so as to urge 
trainees to complete their self-learning to meet the quality demands of training. At the beginning of the training, we 
shall come out with a pre-test for each training module on the basis of training target and traits of curriculum. Before 
every training module begins, trainees are going to finish a corresponding pre-test; and after every training module, a 
post-test should be designed, by analyzing the changes between pre-test and post-test, we may figure out in what extent 
of trainees’ understanding of knowledge and technology has been conveyed to them. In addition, a procedural 
evaluation platform is necessary in the process of training, which makes tracking and evaluating trainees’ study and 
achievement possible. Index of learning evaluation: 1 The results of pre-training test and post-training test; 2 
Participation of online learning; 3 The results of exercises and practice; 4 Assessment of final exams. 

The evaluation of transfer reflects trainees’ capability of applying what they learn into practice. The evaluation of 
transfer needs follow-up investigation on trainees’ practical work after training. Considering the particularity of online 
training, evaluation may be built on the basis of the feedback given by trainees’ colleagues and leaders and trainees’ 
comments published on the forum of the website. 

The evaluation of results mainly evaluates trainees’ progress in performance after training. Because the emersion of 
progress in performance usually takes a period, we evaluate trainees’ progress, and then design suitable post-test 
questionnaires. 

3. Conclusion 

Establishing an effective evaluation system of online training programs is significant to ensure the quality of training 
and keep improvement continuous; which is also necessary to form a mechanism of self-evaluation and supervision, and 
enhance management of quality control system. The evaluation should cover links such as, designing and constructing 
supportive environment and training resources, organizing instructors, setting training curriculum, creating training 
network environment and so on. Exploration in the field of the evaluation system is continuous to ensure the 
effectiveness of the online training programs in a systematical environment, in order to train more high quality 
meteorologists to meet the demand of meteorological and hydrological service. 

References 

Yong WANG & Xiefei Zhi. (2008). Quality Evaluation System of Classroom Training for Meteorologists Based on 
Kirkpatrick Model. Continuing Education, 22(7), 52-54. 

Xingfu Ding. (2005). A New Perspective on Quality in Online Distance Education. Academic Forum, (5), 10-14 

Jing Wang. (2005). New Tendency of Management and Implementation of Distant Training in European Countries. 
Chinese Vocational and Technical Education, (3), 35-36. 

Wang Fusheng et al. (2006). Research on sustainable development of quality-based e-learning: practice and reflection 
on E-learning in Shanghai Jiao Tong University. Practice Exploration, (1),41 -45. 

Yong Wang, Xiefei Zhi, Qing Wu. (2009). Exploration of Building Blended Training Model Based on Blended Learning 
Theory. Continuing Education, 23(6),35-37. 


47 




Vol. 2, No. 4 


International Education Studies 


Table 1. Index system of evaluation 


Index of evaluation 

Topics of questionaires 

1 .Overall reaction to the training 

Topic 1 .Overall Reaction of the training course 

2.Reaction to the online learning environment 

Topic 2.Reaction to the online learning environment 

Online learning Platforms 

Speed of online operation 

Convenience of Operation of the platforms 

Online information resources 

Supportive environment 

3.Reaction to the target of the online training 
program 

Topic 3.Reaction to the target and content of the training program 
The consistency of Content and target 

Richness of content 

Diversity of training activities 

Interactions of online training 

Correlation of the training target and work 

The practicality of Training material 

4.Reaction to the online training content 

5.Participation of the online learning 

Topic 4.participation and gain 

Participation time of online learning per day 

Acceptance of online training 

Factors influencing online learning 

The most impressive part of the training program 

Learning results of training 

6.Reflection on learning results 
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